1. Field of the Invention
The invention relates to accessing segments of a media object over a network. More particularly, the invention provides a system and method for determining a preliminary transmission period for a segment of a media object.
2. Description of the Related Art
Streaming media is a method of making audio, video, and other multimedia data available in real-time, over a network. FIG. 1 illustrates a typical streaming media system. If the source of the streaming media is a live performance, the performance is often recorded as a data signal by an input device, such as, for example, a microphone 102 or a camera 104. The input device then sends the data signal to an encoding system 106 which converts the data signal into a digital form and compresses the digital signal into a streamable data object 107. The streamable data object 107 may be sent to a content creation station 108 for editing or may alternatively be sent directly to a streaming media server 110. The content creation station 108 includes content creation software, such as video editing software, that allows the user to modify the streamable data object 107. The content creation station 108 can also be used to independently create a streamable data object 107. After being modified or created by the content creation station 108, the streamable data object 107 may then be transmitted to the streaming media server 110.
Once the streamable data object 107 is located on the streaming media server 110, the streamable data object 107 is made available for further distribution over a network 100, such as the Internet, to one or more client computers 112. The client computer 112 generally requests access to the streamable data object 107, begins receiving portions of it, and begins rendering the streamable data object 107 into a multimedia presentation for an end user.
One problem, however, related to transmitting streamable data objects over a network is that the client computer 112 must determine when it has sufficient data to begin the presentation or playing of the streamable data object 107. One solution has been to require the client computer 112 to store the whole presentation in memory before beginning the display of the presentation. However, this approach consumes significant system resources and creates unacceptable delay between the user""s initial request for the media and the start of the presentation. Also, the client computer 112 may not have the system capacity to store the whole streamable data object 107.
Under another approach implemented by some existing systems, the client computer 112 starts playing the streamable data object 107 as soon as it is received. Of the problems with this approach, perhaps the most unacceptable is the periodic freezing or delay during playback caused by a lack of sufficient data to render the streamable data object 107. Bandwidth intensive segments of the streamable data object 107 are often responsible for such. To ensure a seamless presentation, the client computer 112 may require more data from the streaming media server 110 than has been transmitted over the network 100. But the amount of data a client computer 112 can receive in any period of time is restricted by the type of network communication it uses.
For example, assuming that the network 100 is the Internet, it is common for a client computer 112 to use a modem to connect to the Internet. However, modems have limited bandwidth capabilities compared to other transmission mediums, i.e., television, radio, Ethernet. The bandwidth of a communication device is defined by the amount of data that can be communicated over a specified time. A 28.8 Kbps (kilobits per second) modem can transmit or receive around 29,000 bits per second.
Depending on the content of a streamable data object 107, the rate at which data is consumed during presentation of the streamable data object 107 may be much higher than the transmission rate for a particular time interval. In such cases, the client computer 112 of a conventional streaming media system simply waits in the middle of the presentation of the streamable data object 107 for the transmission of additional data, creating an unpleasant interruption for the user.
FIG. 2 is a high level block diagram representing the data transmission process of the prior art. Starting at a state 200, the client computer 112 (FIG. 1) requests the streamable media server 110 (FIG. 1) to start transmitting data of the streamable data object 107. Next, at a state 202, the streaming media server 110 starts sending data of the streamable data object 107 (FIG. 1) to the client computer 112. Moving to a state 204, the client computer 112 starts displaying the presentation of the transmitted data. Continuing to a state 206, the client computer 112 stops the presentation in order to wait for further data. Lastly, at a state 208, the client computer 112 finishes displaying the presentation. Although only one interruption is indicated in the flowchart of FIG. 2, the client computer 112 may suffer from multiple interruptions due to having insufficient data at various times in the rendering process.
For further example, referring to FIGS. 3 and 4, a consumption graph for an exemplary rendering of a streamable data object 107 is described. FIG. 4 shows the target bit rate of the client computer 112 in addition to the consumption graph of FIG. 3. The consumption graph illustrates that the number of bits that are consumed during presentation of a streamable data object 107 varies over time. During the first one second interval of the presentation, the client computer 112 (FIG. 1) consumes 30K bits of data. However, during the next second, the client computer 112 consumes only 10K bits of information. Table 1, based on the consumption graph in FIGS. 3 and 4, further illustrates a situation wherein the total number of bits that are needed by the client computer 112 to maintain an uninterrupted playback is larger than the total number of bits it has received at multiple points in time.
As can be seen from Table 1 and FIG. 2, after one second, in order to seamlessly present the streamable data object 107, the client computer 112 needs 30K bits; however, the client computer 112 has only received 20K bits of the streamable data object 107. Therefore, the client computer 112 has to wait for further data before starting the presentation. Further, after the presentation is started, the client computer 112 has to halt again because the presentation requires 70K bits of data at three seconds, but the client computer has only received 60K bits. This halt manifests itself, in existing systems, by freezing the presentation.
To attempt to overcome this difficulty, some systems transmit a portion of a streamable data object before playback starts. However, all multimedia presentations are not always played from the beginning, and transmitting an initial portion of a streamable data object does not help to address such situations.
The client computer 112 typically has a rendering program (not shown) that allows the user to xe2x80x9cseekxe2x80x9d to the middle or any number of secondary positions in the presentation before or during playback. After starting to play at the secondary position, the client computer 112 may require more data for the presentation than has been transmitted. Thus, the client computer 112 must pause and wait for the transmission of additional data. One solution to this problem has been to estimate the initial transmission buffer for the secondary position as being equal to or slightly greater than the initial transmission buffer for the whole streamable data object. However, this approach is merely a rough estimate and cannot guarantee that the client computer 112 will render the presentation without further interruptions and additional waits for further data. Thus, this approach does little to ensure a smooth, uninterrupted playback of a media object.
Variations in rendering requirements (data consumption over time) are more apparent in low bandwidth media systems, such as MacroMedia Shockwave Flash 2.0. MacroMedia Shockwave Flash 2.0 is one type of content creation system that generates television-like cartoons for business, entertainment or education. In these low bandwidth media systems, each of the graphical objects (xe2x80x9ccharactersxe2x80x9d) of the presentation are well defined, and multimedia presentations are often developed wherein these characters move from segment to segment or from frame to frame. Accordingly, the rendering requirements of the presentation vary greatly, generally becoming quite high when a new character is introduced. Due to this variation, the amount of data that would ensure uninterrupted playback if received prior to rendering the streamable data object 107 from its beginning, may differ widely from the amount needed to ensure uninterrupted playback from a secondary position, such as the middle of the streamable data object 107.
Therefore, there is a need for a system and a method that provides for the uninterrupted presentation of streamable data objects, including those streamable media objects imposing rendering requirements that vary over time. Moreover, the system and the method should not require the client computer 112 to store the entire streamable date object before starting the presentation. Further, the system and the method should provide for the uninterrupted rendering of a streamable data object 107 for each of the possible playback positions of the streamable data object 107.
The system and method of the present invention have several features, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of this invention as expressed by the claims that follow, its more prominent features will now be discussed briefly. After considering this discussion, and particularly after reading the section entitled xe2x80x9cDetailed Description of the Inventionxe2x80x9d one will understand how the features of this invention provide several advantages over traditional streaming systems.
The invention includes a streaming media system that provides for the calculation of a pre-roll for each segment of a multimedia data object. Using the calculated pre-roll, a client computer can render a multimedia presentation starting at each playable position of the multimedia data object with interruption. Further, the streaming media system of the invention minimizes the amount of buffering that is needed before rendering a presentation based upon the multimedia data object.
Further, the streaming media system of the invention has a graphical utility that enables a user to re-arrange the presentation of the multimedia object such that the pre-roll of the multimedia data object is minimized.
One embodiment of the present invention is a method of calculating a pre-roll for a plurality of segments in a multimedia data object. The method comprises identifying a plurality of segments in the multimedia data object, calculating a pre-roll for each of the plurality of segments. An aspect of the embodiment further comprises storing the pre-roll for each of the plurality of segments in the multimedia data object. Another aspect of the embodiment is that wherein calculating a pre-roll includes determining a target bit rate of a client computer. A further aspect of the embodiment is one wherein calculating a pre-roll includes determining the size of each of the plurality of segments, and wherein the pre-roll for each of the plurality of segments corresponds to a minimum amount of data to send to the client computer to enable the client computer to render the multimedia object without interruption starting from the respective segment. Still another aspect of the embodiment is one wherein the data of each segment is examined only once.
Another embodiment of the invention a system for calculating a pre-roll for a plurality of segments in a multimedia data object. The system comprises means for identifying a plurality of segments in the multimedia data object and means for calculating a pre-roll for each of the plurality of segments, the pre-roll for each of the plurality of segments corresponds to a minimum amount of data to send to a client computer to enable the client computer to render the multimedia data object without interruption starting from the respective segment. In another aspect, the embodiment further comprises means for generating the multimedia data object. In still another aspect, the embodiment further comprises means for rendering the multimedia data object into a multimedia presentation. In yet another aspect, the embodiment further comprises means for displaying the pre-roll. In one more aspect, the embodiment further comprises means for storing the pre-roll for each of the plurality of segments in the multimedia data object.
Yet another embodiment of the invention is a system for streaming a multimedia data object across a network. The embodiment comprises (1) a streaming media server operably connected to the network; (2) a client computer operably connected to the network; and (3) a multimedia data object including a segment index, the segment index defining a pre-roll, the pre-roll indicating a transmission of data from the streaming media server to the client computer that occurs before the client computer displays a presentation that is represented by the multimedia data object, the presentation starting at a selected segment of the multimedia data object. Another aspect of the embodiment further comprises a pre-roll calculation program that creates the segment index. Still another aspect of the embodiment is one wherein the selected segment is a frame.
Still another embodiment of the invention is a method of analyzing a multimedia data object that is adapted for rendering on a computer. The method comprises determining the rendering requirements of at least a portion of a multimedia data object; and displaying the rendering requirements of the multimedia data object. Another aspect of the embodiment is one wherein displaying the rendering requirements further includes plotting a consumption graph. Still another aspect of the embodiment is one wherein determining the rendering requirements of at least a portion of the multimedia data object includes identifying a plurality of segments within the multimedia data object; and determining the amount of data within each of the segments of the multimedia data object.
One more embodiment of the present invention is a method of analyzing a multimedia data object that is adapted for rendering by a computer. The method comprises (1) determining a communication bit rate of a second computer; (2) determining the rendering requirements of the second computer for displaying at least a portion of the multimedia data object; (3) determining a pre-roll for said portion of the multimedia data object, the pre-roll corresponding to a minimum amount of data to send to the second computer to enable the second computer to render the multimedia object without interruption starting from the portion; and (4) displaying the pre-roll. Another aspect of the embodiment is one wherein the multimedia data object has a plurality of segments, wherein the method for determining the rendering requirements further includes determining the rendering requirements for each segment of the multimedia data object, and wherein determining the pre-roll further includes determining a pre-roll for each segment of the multimedia data object. Yet another aspect of the embodiment further includes storing the pre-roll. Still another aspect further includes re-organizing the multimedia data object.
Still another embodiment of the present invention a program storage device storing instructions that when executed by a computer perform a method. The method comprises (1) identifying at least two positions within a multimedia data object; (2) determining a communication bit rate of a second computer; (3) determining the rendering requirements of the second computer for rendering the multimedia data object at each of the at least two positions; (4) determining a pre-roll for each of the at least two positions within the multimedia data object; and (5) storing the pre-roll for each of the at least two positions.
Another embodiment of the present invention is a method of streaming a multimedia data object across a network. The method comprises (1) receiving a request for the transmission of a selected portion of a multimedia data object; (2) determining a pre-roll for the selected portion of the multimedia data object, the determining based in part on the rendering requirements of the multimedia data object and the transmission rate of the a client computer; (3) transmitting the multimedia data object to a client computer; and (4) signaling the client computer to start rendering the multimedia data object. In another aspect of the embodiment, the method further comprises rendering the multimedia data object.
Still another embodiment of the present invention is a multimedia data object stored on a computer readable medium. The multimedia data object comprises a plurality of segments; and a segment index, the segment index indicating a pre-roll for each of the plurality of segments. One aspect of the embodiment is one wherein the segment index further comprises the position of each segment within the multimedia data object. Another aspect of the embodiment is one wherein each of the segments is a frame.
Another embodiment of the present invention is a streamable multimedia object stored on a computer readable medium. The streamable multimedia object comprises a multimedia file having a sequence of segments; and a table having an entry for each of a plurality of said segments, each of said entries indicating a minimum amount of data of the multimedia file that must be sent to a client computer before the client computer begins rendering, to enable the client computer to render the multimedia file uninterrupted starting at a segment corresponding to the respective entry, the client computer receiving data at a given bit rate.
One more embodiment of the present invention is a method to facilitate the uninterrupted play of a multimedia file having a sequence of segments by a client computer over a narrow bandwidth network. The method comprises the acts of identifying a plurality of segments within said multimedia file; and determining, for each of said plurality of segments, the minimum amount of data of the multimedia file that must be sent to a client computer before the client computer begins rendering, to enable the client computer to render the multimedia file uninterrupted starting at a respective one of each of said segments, the client computer receiving data at a given bit rate.
Still one more embodiment of the present invention is a multimedia streaming system. The system comprises (1) a network; (2) a multimedia data object; (3) a streaming media server operably connected to the network and the multimedia data object, the streaming media server adapted to stream the multimedia data object over the network; (4) a client computer operably connected to the network, the client computer adapted to receive the multimedia data object over a network, the client computer having a rendering program that is adapted to render the multimedia data object into a multimedia presentation; and (5) a pre-roll calculation program that generates a segment index, the segment index defining a pre-roll, the pre-roll indicating a data transmission that occurs before the client computer renders a multimedia presentation represented by the multimedia data object, the presentation starting at a selected segment of the multimedia data object.